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Detection  of  a  PSK  Signal  Transmitted  Through 

a  Hard-Limited  Channel 

PRAVIN  C.  JAIN  and  NELSON  M.  BLACHMAN 


Abstract — This  paper  considers  the  problem  of  the  detection  of  a 
binary  phase-modulated  carrier  which  has  been  transmitted  along  with 
noise  through  a  hard-limiting  repeater,  corrupted  by  additional  noise, 
and  demodulated  by  cross  correlation  and  sampling  at  the  receiver.  Three 
equivalent  expressions  are  obtained  for  the  error  probability  of  the 
receiver  output.  Two  of  these  expressions  take  the  form  of  an  infinite 
series  involving  either  confluent  hypergeometric  functions  or  modified 
Bessel  functions.  A  third  form  allows  representation  of  the  error  prob¬ 
ability  in  terms  of  Rice’s  Ie  function.  In  the  special  case  where  the  signal- 
to-noise  ratios  p j2  and  p22  at  the  limiter  and  receiver  inputs  are  equal, 
the  error  probability  is  simply 

Pe  =  i  exp  (-pS). 

Asymptotic  expressions  for  the  error  probability  for  large  and  small 
signal-to-noise  ratios  are  also  derived.  The  error  probability  is  found  to 
be  smaller  than  that  of  a  linear  system  for  all  practical  values  of  limiter 
and  receiver  input  signal-to-noise  ratios.  The  optimum  repeater  non¬ 
linearity  is  investigated  and  is  shown  to  be  a  limiter  at  such  signal-to-noise 
ratios. 
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I.  Introduction 

HE  MODEL  for  the  communication  system  to  be 
considered  in  this  paper  is  shown  in  Fig.  1.  A  binary 
phase-shift-keyed  (PSK)  signal  is  assumed  to  get  from  the 
transmitter  to  the  receiver  via  a  hard-limiting  repeater  in 
the  transmission  channel.  Additive  Gaussian  noise  is  in¬ 
troduced  on  both  the  uplink  (repeater  noise)  and  the  down¬ 
link  (receiver  noise).  Detection  of  the  signal  at  the  receiver 
is  accomplished  by  a  cross  correlation-and-sampling  op¬ 
eration.  The  receiver  is  assumed  to  heterodyne  the  received 
signal  with  a  correctly  phased  local  reference  and  to  base  its 
decision  on  the  sign  of  the  baseband  output  of  a  zonal  low- 
pass  filter  that  is  sampled  once  during  each  data  bit. 

The  results  for  this  single-sample  detection  model  can 
be  extended  to  include  the  effect  of  postdetection  integration 
over  the  full  bit  duration  which  is  customary  in  correlation 
detection  of  binary  PSK  signals,  via  the  commonly  assumed 
multiple  sampling  and  majority  decision  [1]— [3].  This  is 
achieved  by  approximating  the  integration  operation  by  a 
sum  of  TW  independent  receiver-output  samples  taken  at 
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TRANSMISSION  CHANNEL 


RECEIVER 


x(t)  cos  -  y(t)  sin  wQt  u<t)  cos  w0t  v(t)  sin  u)Qt 


Fig.  1.  Block  diagram  of  PSK  system. 


the  Nyquist  rate  and  supposing  that  on  the  basis  of  these 
samples,  TW  independent  decisions  are  made  on  each  bit; 
a  final  overall  decision  is  then  taken  on  that  bit  by  a  majority 
vote.  It  is  preferable  that  TW— the  product  of  the  bit  dura¬ 
tion  times  the  signal  bandwidth — be  odd.  The  probability 
of  error  PE  is  then  equal  to  the  probability  that  more  than 
half  the  decisions  will  be  in  error  and  is  given  by  the 
binomial  distribution 


Pe 


(tw-dh 

I 

k~0 


Pe)k  Plw~k 


(1) 


where  Pe  is  the  probability  of  error  for  any  one  sample. 
Jacobs  [1]  has  plotted  PE  as  a  function  of  TW{\  -  Pe~\2 
for  various  values  of  TW.  In  this  paper  we  determine  Pe 
on  the  understanding  that  the  error  probability  with  post¬ 
detection  integration  can  then  be  calculated  from  (1)  or  by 
using  the  plots  given  by  Jacobs.  Since  PE  is  a  monotonic 
function  of  Pe,  the  conclusions  reached  in  this  paper  con¬ 
cerning  the  helpfulness  of  limiting  with  single-sample  detec¬ 
tion  also  apply  with  postdetection  integration.  This  is  in 
agreement  with  the  results  and  conclusions  of  Davisson  and 
Milstein  [2]  and  Lyons  [3]. 

The  calculation  of  the  error  rate  with  a  hard-limiting 
repeater  is  of  considerable  interest  both  because  limiters  are 
often  used  at  the  inputs  to  traveling-wave-tube  amplifiers 
when  maximum  output  power  is  demanded  of  them  and 
because  at  reasonably  high  signal-to-noise  ratios,  as  we  shall 
show,  for  a  given  average  repeater  output  power,  the  error 
probability  is  smaller  with  a  limiting  repeater  than  with  a 
linear  one.  We  shall  also  see  that  a  limiter  is  the  optimum 
repeater  nonlinearity  at  high  signal-to-noise  ratios.  Since 
this  superiority  of  the  limiter  seems  to  strike  some  com¬ 
munication  theorists  as  incredible,  we  begin  with  some 
heuristic  comments  on  this  matter. 

A.  The  Desirability  of  Limiting 

Although  cross  correlation  detection  of  binary  PSK 
signals  in  noise  is  optimum,  it  is  optimum  only  among  the 
things  that  can  be  done  solely  at  the  receiver.  In  the  present 
context  optimization  would  have  to  include  the  best  signal 
processing  by  the  repeater  for  a  given  average  repeater 
output  power.  Because  keying  simply  switches  the  carrier 
phase  between  0  and  180°,  the  repeater  should  waste  no 
power  on  the  quadrature  component  of  the  noise  and  should 


put  all  of  its  power  into  retransmitting  the  inphase  com¬ 
ponent  of  the  signal-plus-noise,  which  is  a  sufficient  statistic 
in  regard  to  mark-versus-space  decisions.  The  optimum 
repeater  will  therefore  cross  correlate,  decide,  and  regenerate 
the  signal  corresponding  to  its  decision. 

It  is  easy  to  see  that  such  a  signal-regenerating  repeater 
will  yield  performance  superior  to  that  of  a  linear  repeater. 
For  example,  if  the  up  and  downlink  signal-to-noise  ratios 
are  both  reasonably  high  and  are  equal,  the  net  overall 
signal-to-noise  ratio  at  the  receiver  with  a  linear  repeater 
will  be  3  dB  less,  making  the  error  probability  many  times 
that  of  a  regenerating  repeater,  where  the  error  probability 
is  merely  doubled  by  the  two  decisions  in  tandem.  In  fact, 
if  we  compare  a  chain  of  linear  repeaters  with  a  chain  of 
regenerating  repeaters  along  the  path  from  the  transmitter 
to  the  receiver  with  the  product  of  their  signal-plus-noise- 
to-signal  ratios  held  constant  (being  equal  to  the  overall 
net  signal-plus-noise-to-signal  ratio  for  the  linear  chain),  we 
see  that  the  error  probability  remains  fixed  for  the  linear 
system  as  the  number  of  repeaters  is  increased.  However,  it 
approaches  zero  for  the  regenerating  repeaters,  as  the  signal- 
to-noise  ratio  for  each  repeater  becomes  proportional  to  the 
number  of  repeaters,  thus  reducing  the  error  probability  for 
each  to  a  very  small  value — going  to  zero  even  when 
multiplied  by  the  number  of  repeaters. 

At  high  signal-to-noise  ratios  the  inphase  output  com¬ 
ponent  of  a  limiter  will  be  very  nearly  the  same  as  that  of  a 
regenerating  repeater,  but  it  is  not  possible  to  get  a  vanishing 
error  probability  with  a  chain  of  limiters,  since  the  phase 
errors  of  their  outputs  (due  to  the  quadrature  noise)  will 
accumulate  in  exactly  the  same  way  that  the  inphase  com¬ 
ponent  of  the  noise  builds  up  in  a  chain  of  linear  repeaters. 
An  error  results  with  limiters  when  the  quadrature  noise 
components  take  the  signal  phasor  a  quarter  of  the  way  or 
more  around  a  circle  about  the  origin,  whereas  in  the  linear 
case  an  error  occurs  when  the  inphase  noise  totals  more 
than  the  length  of  the  signal  phasor  and  is  opposite  in  sign. 
Since  a  90°  arc  is  n/2  times  as  long  as  the  radius,  limiting 
provides  an  improvement  of  nearly  20  log,0  njl  =  3.82  dB 
over  a  linear  system — “nearly”  because  a  90°  phase  change 
in  either  direction  will  produce  an  error.  Having  established 
the  plausibility  of  an  error  probability  reduction  due  to 
limiting  at  high  signal-to-noise  ratios,  we  now  proceed  to 
treat  this  topic  more  thoroughly  by  means  of  a  detailed 
mathematical  analysis. 
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In  Section  II  we  will  obtain  first  an  expression  for  the 
oxftput  of  the  zonal  low-pass  filter  in  Fig.  1  and  then  derive 
three  equivalent  expressions  for  the  error  probability  in¬ 
cluding  asymptotic  results  for  large  and  small  signal-to-noise 
ratios.  Numerical  results  for  the  error  probability  are  pro¬ 
vided  in  Section  III,  and  the  performance  of  a  hard-limited 
system  is  compared  with  that  of  a  linear  repeater  system. 
The  investigation  of  the  optimum  repeater  nonlinearity  will 
be  the  topic  of  Section  IV,  with  conclusions  provided  in 
Section  V. 


II.  Mathematical  Analysis 

The  communication  system  shown  in  Fig.  1  has  been 
recently  investigated  by  Jain  [4]  and  Davisson  and  Milstein 
[2].  Our  treatment  in  this  paper  will  simplify  and  extend 
some  of  the  derivations  and  results  obtained  in  [4]  and  [2], 
as  well  as  many  new  results  including  entirely  new  material 
on  the  optimization  of  the  nonlinearity. 


A.  Calculation  of  Receiver  Output 

The  input  to  the  limiter  in  Fig.  1  consists  of  a  binary 
phase-modulated  signal  of  frequency  f0  and  amplitude  A 
plus  zero-mean,  stationary  Gaussian  noise  of  rms  value  a1 . 
The  bandpass  filter  preceding  the  limiter  is  assumed  to  be 
wide  enough  to  pass  the  signal  with  negligible  distortion  and 
to  limit  the  uplink  noise  to  a  bandwidth  that  is  small  com¬ 
pared  with  the  center  frequency  of  the  filter.  The  limiter 
input  may  be  expressed  as 

fi(t)  =  A  cos  [a V  +  0(f) ]  +  x(t)  cos  co0t  —  y(t)  sin  co0t 
=  *(0  cos  [co0t  +  0(f)]  (2) 

where  the  envelope  R(t)  and  the  phase  0(f)  are  given  by 
R(t)  =  \I\A  cos  0(f)  +  x(f)]2  +  y\t) 


0(f)  =  arctan 


y(  0 

A  cos  0(f)  +  x(f)  ’ 


(3) 


During  any  bit  interval,  0(f)  is  either  0  or  n,  depending 
upon  whether  a  mark  or  space  is  being  transmitted. 

The  bandpass  limiter  is  assumed  to  be  ideal  in  the  sense 
that  its  fundamental-zone  output  /0(f)  is  given  by 


/o(f)  =  cos  [a>0f  +  0(f)]  (4) 

i.e.,  envelope  variation  has  been  completely  removed 
without  distorting  the  phase  modulation. 

The  signal  is  then  transmitted  to  the  receiver  and  noise  is 
added  to  it  on  the  downlink  in  the  receiver  front  end.  We 
assume  this  noise  is  stationary,  zero-mean,  Gaussian  of  rms 
value  <t,  and  is  independent  of  the  uplink  noise.  Hence,  the 
receiver  input  may  be  expressed  as 


g{t)  —  acos[<y0f  +  0(f)]  -f  w(f)  cos  co0t  —  v(t)  sin  co0t.  (5) 

The  amplitude  a  of  the  signal  is  determined  by  the  amplifica¬ 
tion  in  the  repeater  and  the  downlink  losses  in  the  link 
following  the  limiter. 

At  the  receiver  the  signal  g(t )  is  multiplied  by  the  correctly 
phased  reference  2  cos  co0t  and  then  low-pass  filtered  to 


remove  the  double-frequency  components.  The  filter  output 
is  given  by 

z(t)  =  a  cos  0(f)  +  u(t ).  (6) 


B.  Calculation  of  Error  Probability 

At  the  receiver  output,  the  decision  as  to  whether  a  mark 
or  a  space  was  transmitted  is  made  on  the  basis  of  whether 
the  filter  output  at  the  sampling  instant  is  positive  or  neg¬ 
ative.  In  order  to  calculate  the  probability  of  error  of  the 
decision,  we  need  to  determine  the  probability  density  func¬ 
tion  q{z)  of  the  filter  output.  This  is  obtained  directly  from 
(6)  as  the  convolution  of  the  probability  density  function 
p(0)  of  the  phase  and  a  Gaussian  density  function  for  the 
inphase  downlink  noise.  The  result  is 

<7> 

where  a2  is  the  noise  power  at  the  receiver  input.  This 
integral  representation  for  q(z)  will  be  convenient  for  the 
calculation  of  the  error  probability. 

We  assume  that  each  transmitted  symbol  is  statistically 
independent  of  its  predecessors  and  that  mark  and  space 
signals  are  equally  probable.  Hence,  since  the  noise  at  the 
low-pass  filter  output  has  a  zero-mean  value,  the  probability 
of  error  Pe  of  a  decision  is  equal  to  the  error  probability 
for  either  a  mark  or  a  space  signal.  Thus 

Pe  =  Pem  =  PeM-  (8) 

The  probability  of  error  in  detecting  a  mark  (0  =  0)  is 
equal  to  the  probability  that  the  filter  output  will  be  negative 
at  the  sampling  instant,  viz., 

pe=  \  <l(z)  dz.  (9) 

J  -  oc 

Substituting  the  expression  for  q(z)  in  (9)  and  changing  the 
order  of  integration  yields 


f°  /  —  (z  —  a  cos  0)2\  ,  ]  , ,  .... 

l.exp( — 2^-r.K  (10> 


The  integral  in  the  brackets  can  be  expressed  in  terms  of  the 
error  function  [5],  Thus  (10)  may  be  written  as 


Pc 


-if  [l-erffecos0)]^)# 

erf  (videos  p(4>)  d(f>.  (11) 


Clearly,  with  no  noise  on  the  uplink,  the  error  probability 
is  unaffected  by  the  presence  of  the  limiter  and  is  simply 

Pe  —  i  —  i  erf  (a/yfl  cr).  Likewise,  with  no  noise  in  the 
downlink  the  limiter  will  have  no  effect  on  Pe.  This  is  easily 
seen  by  expressing  cos  0(0  in  (6)  as  [A  +  x(t)]/R(t),  and 
noticing  that  Pe  is  the  same  as  the  probability  of  the  inphase 
component  x(t)  of  the  uplink  noise  exceeding  the  signal 
amplitude  A ,  which  is  simply  Pe  =  \  \  erf  {Ajyjl  cr,). 
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To*  evaluate  the  integral  in  (11),  we  use  the  representation 
for  the  error  function  [6] 

.  ,  l  a  ...A  2  p0  sin  (va  cos  <f>)  , 


erf  (  “t  cos  4> i 

=  2  f 

V  v  2  a  / 

n  Jo 

which  is  seen  by  Plancherel’s  theorem  to  be  correct.  This 
representation  is  convenient  for  expressing  the  error  func¬ 
tion  in  the  form  of  a  Fourier  series.  Using  the  expression 

sin  (va  cos  </>)  =  2  £  (- l)nJ2n+  x(va)  cos  (In  +  1  )<p  (13) 


we  find  that  (12)  becomes 


erf  (-L -  cos  *)=-£(-  D”  f  J2"+l(Va) 

\n/2  a  /  .  n  n= o  J0  v 

■  exp  ^ ^ v  cos  (2n  +  !)</>•  (14) 


The  integral  in  (16)  may  now  be  evaluated  with  the  aid  of 
(17).  The  result  is  [10] 


The  integral  here  is  attributed  to  Weber  and  Sonine,  and  its 
solution  is  given  in  terms  of  confluent  hypergeometric 
series  [7],  The  result  is 

erf  (— p—  cos  (j) ) 

'  v2  a  > 

=  2  f  (-l)T(n  +  j)  /  a  \  2,1+1 
n  »=o  (2 n  4-  1)!  \ y  2  a' 

•  jfi  t  #  2n  +  2,  -  cos  (2n  +  1  )<f>  (15) 

where  ,Fj(-)  is  the  confluent  hypergeometric  series  and 
F(-)  is  the  gamma  function.  Substituting  (15)  in  (11)  yields 


cos  (2 n  +  1  )<t>p(<5>)  d<j> 
(  A  \2n+1 

\S.aJ 


V 

’  w) 


p  =  i  _  1  y  (-D'’-r(n  +  1)  /  a  \ 2,1+1 

c  2  n  n=o  (2 n  +1)!  \\/2  o' 

'lFl  ("+{’2n  +  2’  -  J?) 

•  f  cos  (2/i  +  1  )(j)  *  p(4>)  d(f).  (16) 

Jo 

A  number  of  equivalent  representations  for  the  prob¬ 
ability  density  function  p(4>)  are  known  in  the  literature 
[8],  [9].  They  would  lead  to  equivalent  but  different  ex¬ 
pressions  for  the  error  probability.  In  this  paper  we  shall 
consider  two  different  representations  for  p(4>)  and  derive 
the  corresponding  expressions  for  the  error  rate.  First,  we 
consider  the  integral  representation  for  p(4>)-  When  a  mark 
is  transmitted,  i.e.,  6(t)  =  0,  p(4>)  is  given  by  [9] 


m  „  a 

2.71(7  j 


R  exp 


—  (R2  —  2AR  cos  4 f)] 


.  r(w_+j)  F  L  +  1  2w  +  2,  (18) 

(2  n  +  1)!  V  2  20-!2/ 

Substituting  (18)  in  (16),  we  obtain  the  probability  of  error 

P  _  1  _  1  V  ±  ± j>  -  -2.+  1 

e  2  n,k  [(2n  +  l)!]2 

‘  iF i(n  +  2n  +  2,  —  Pi2) 

'  iF  i(n  +  2 n  +  2,  —  p22)  (19) 

where  p2  =  A2  (2a2  =  limiter  input  or  uplink  signal-to- 

noise  ratio,  and  p22  =  a2/2<r2  =  PJo2  =  receiver  input  or 
downlink  power-to-noise  ratio.  Note  that  P,  contains  both 
signal  and  retransmitted  limiter  input  noise  power.  We  now 
show  that  the  series  in  (19)  is  convergent.  Since 

|1F1(a,t,  —  x)|  <1,  for  b  >  a  >  0  <  x  (20) 

the  series  in  (19)  will  converge  if  the  dominating  series 
obtained  by  setting  both  confluent  hypergeometric  functions 
equal  to  one  converges.  The  resulting  series  is  absolutely 
convergent  for  all  values  of  pt2  and  p22,  as  can  be  shown 
easily  by  d’Alembert’s  ratio  test  for  absolute  convergence 

Dm  Ssti  -  lim  -  ("  4 1  jX"  ' ±  (p,p;)!  -  0. 

n-oc  U„  (2n  +  3)2  •  (2n  +  2)2 

(21) 

An  alternative  expression  for  Pe  is  obtained  by  using  the 
relationship 

iFj(n  +  i,  2 n  +  2,  -p2) 

~ _ /  .  „i  .  ~>2nr  /  Jl\ 


exp  (-p2/ 2)  •  n! 

•  22" 

p2" 

_ 

where  /„(*)  is  the  modified  Bessel  function. 
The  expression  for  Pe  then  becomes 

i  ~  „  r  2  ,  n  « 


Px  *  Pi 


-(PA  +  P2  ) 
2 


f  (-D" 

-o  2n  +  1 


f-(¥)  + '■*•(¥) 
■■  (¥)*<-■(*) 


Next  we  derive  an  expression  for  the  error  rate  by  using 
a  closed-form  representation  for  the  probability  density 
function  of  the  phase  [9] 


dR.  (17) 


m  -  + 

271 


—  cos  <p  exp  (— p2  sin2  <f> ) 

An 

•  [1  +  erf  (pi  cos  <£)].  (24) 
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Substituting  (24)  in  (11)  and  performing  the  integration 
provides  an  expression  for  the  error  probability  in  terms  of 
Rice’s  Ie  function  [11] 


where 


Pe  =  i[l  - 


Pi2  +  P22 


k2  Ie(k,x)~] 


Pi2  +  P2 


k,x)  =  r 

J  n 


exp  ( —  t)  m  I0(kt )  dt. 


When  pi  =  p2,  le(0,x)  =  1  -  exp  (-x),  and  the  error 
probability  is  then  simply 

Pe  =  iexp(-p,2).  (28) 

C.  Asymptotic  Values  for  the  Error  Probability 

It  is  interesting  to  examine  the  behavior  of  Pe  for  small 
and  large  SNRs.  An  asymptotic  result  for  p2  and  p2  much 
less  than  unity  is  obtained  by  retaining  only  the  first  term 
of  the  infinite  series  in  (19)  and  the  first  term  of  the  confluent 
hypergeometric  series  appearing  in  it 

Pe  ~  iO  -  P1P2),  for  ptp2  «  1.  (29) 

When  both  pf  as  well  as  p2  -  p22  are  large,  we  can 
find  an  asymptotic  expression  for  Pe  by  using  the  expansion 
of  Ie(k,x )  for  large  jc  [11] 


/e(/c,co) 


V 1  —  k2 
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in  (25).  We  thus  obtain 


[1  -  erf  p2] 


Linear  System 

-  Hard-Limited  System 


2  .  J  •  "  \  - 

exp(-p/)  forpi>p2>>1  \ 

2 P2  s/n(Pl2  -  p22)  1  _  \ 

and  p^  -  p22  »  1.  (32)  \  \ 

When  pf2  is  very  much  larger  than  p22,  (31)  reduces  to  -  \  s~ 

pe  =  t[1  -  erf  p2],  for  p,  ^  00  (33)  Linear  Sys,em  ~ 

which  is  the  expected  result  for  ideal  limiting  in  the  absence  Hard-Limited  system 

of  uplink  noise.  Because  Pe  is  a  symmetric  function  of  pj2  _  _ 

and  p22,  the  corresponding  results  for  p22  >  p2  are  ob-  °° 

tained  by  simply  interchanging  p2  and  p22  in  (31)-(33).  1Q-6  1  ,  ,  ,  ,  ,  , 

III.  Numerical  Results  24  *18  ’12  "6  0  6  12  18  24 

The  results  of  numerical  evaluation  of  the  error  rate  are  L'MITER  'NPUT  SIGNA1-T0'N0'SE  RAT,°  P f  ~  dB 

shown  in  Fig.  2  as  a  function  of  p2  for  various  values  of  Fig-  2-  PSK  error  rate  as  function  of  limiter  input  signal-to-noise 
p22.  Similar  curves  for  Pe  as  a  function  of  p22  with  Pl 2  as  rat'°  f°r  C°nStan* vaIueS  of  received  P°wer-to-noise  ratio- 

the  parameter  are  obtained  by  merely  interchanging  the 
labels  pf1  and  p22  in  Fig.  2,  since  the  expression  for  the 
error  rate  is  symmetric  in  p22  and  p22.  Both  (19)  and  (23) 
were  programmed,  and  the  results  obtained  were  identical. 


-24  -18  -12  -6  0  6  12  18  2 

LIMITER  INPUT  SIGNAL-TO-NOISE  RATIO  P2  —  dB 
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as  expected.  Equation  (25)  was  not  used,  because  we  had 
not  yet  discovered  it.  When  both  py2  and  p22  are  large, 
computation  of  the  error  rate  from  either  (19)  or  (23) 
becomes  difficult,  and  it  is  important  that  the  series  in  (19) 
and  (23)  be  computed  very  accurately.  We  found  that  (23) 
was  easier  to  use  than  (19)  at  low  error  rates  because  the 
computation  of  the  Bessel  functions  in  (23)  requires  fewer 
terms  of  their  power  series  expansion  than  do  the  two 
hypergeometric  functions  appearing  in  (19).  The  results 
obtained  by  using  the  asymptotic  expression  (32)  for  Pe 
when  p  2  and  p2  are  both  large  were  found  to  be  in  ex¬ 
cellent  agreement  with  the  numerical  evaluation  of  (23). 

The  computation  of  the  error  rate  from  (25)  would  re¬ 
quire  accurate  calculation  of  the  Ie  function  defined  by  (27). 
Rice’s  table  [11]  for  the  Ie  function  does  not  give  sufficient 
coverage  for  calculation  of  error  rates  under  conditions  of 
practical  interest.  The  Ie  function  can  be  calculated  by 
either  numerical  integration  of  (27)  or  from  the  series  given 
by  Rice  [11] 


For  a  hard-limiting  repeater  the  corresponding  expression 
is  given  by  (29).  Thus  we  see  that  at  low  SNRs  the  linear 
repeater  yields  better  performance  than  the  hard  limiter 
(although  both  do  badly).  The  latter  requires  a  1.05-dB 
increase  in  the  product  of  the  two  SNRs  to  obtain  the  same 
error  probability. 

Fig.  2  shows  a  bottoming  of  the  error  rate  for  both  a 
linear  and  a  hard-limiting  system  representing  an  irreducible 
error  probability  that  depends  on  the  noise  present  either  at 
the  repeater  or  the  receiver  front  end,  depending  upon 
whether  the  abscissa  is  p2  or  p2.  It  is  seen  in  Fig.  2  that, 
as  p2  tends  to  infinity,  both  systems  tend  to  the  same 
limit  given  by  (33).  The  significant  difference,  however,  is 
that  a  hard-limiting  system  approaches  the  limit  much 
faster,  i.e.,  at  lower  values  of  p2,  than  a  linear  system  for 
any  given  value  of  p  2 . 

At  large  values  of  p2  and  p2  the  error  rate  for  the  linear 
system  is  obtained  by  using  the  first  term  of  the  asymptotic 
expansion  for  the  error  function  in  (34) 
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A.  Comparison  with  Linear-Repeater  Performance 

The  dashed  lines  in  Fig.  2  show  the  error  rate  for  a  linear 
system,  i.e.,  a  system  with  the  limiter  replaced  by  a  linear 
repeater  having  the  same  average  output  power  as  the 
limiting  repeater  in  Fig.  1.  The  error  probability  for  a  linear 
system,  which  has  a  Gaussian  noise  distribution  at  the 
receiver  output,  is 

Pe  =  i[l  -  erf(p/v/2)]  (34) 


where  p2  is  the  receiver  output  SNR.  The  repeater  output 
power  Pt  =  a2! 2  is  divided  between  the  signal  and  the 
uplink  noise  in  proportion  to  their  power  levels  at  the 
repeater  input.  At  the  receiver  the  received  waveform  is 
processed  by  using  correlation  detection  with  baseband 
zonal  filtering  and  sampling.  The  correlation  operation 
suppresses  the  quadrature  components  of  both  retransmitted 
uplink  and  downlink  noise,  thereby  providing  a  3-dB  im¬ 
provement  in  the  SNR  at  the  receiver  output  over  the  SNR 
at  its  input.  The  output  SNR  for  the  linear  system  is  thus 


p2  =  2 


PiW 


1  +  Pi  +  p2 


(35) 


Using  two  terms  of  the  power-series  expansion  of  the 
error  function  in  (34)  and  approximating  (35)  as  p  = 
V 2  Pip2,  we  find  that  the  error  probability  at  low  SNRs 
with  a  linear  repeater  is  approximately 


(36) 


for  large  pi  and  p2.  (37) 

The  dominant  factor  here  is  the  exponential,  while  in  the 
case  of  the  limiter  with  px  >  p2  it  is  exp  (— p22)  as  in  (28) 
and  (32).  Comparing  the  respective  exponents,  we  see  that 
the  limiter  affords  an  SNR  advantage  of  1  +  p2\p  2  <  2, 
i.e.,  at  most  3  dB  in  each  of  the  two  links. 

If  the  larger  of  the  two  SNRs,  px2,  is  increased  while 
the  smaller  one,  p22,  is  held  fixed,  then  we  would  obtain  the 
same  exponent  for  the  linear  system  in  (37)  as  with  the 
limiter  when  px2  is  infinite.  In  this  sense  the  limiter  ex¬ 
hibits  an  unbounded  SNR  advantage.  For  example,  we  see 
from  Fig.  2  that  for  =  2.4  x  10“ 4  with  p2  =  8  dB 
there  is  a  9-dB  horizontal  spread  in  the  value  of  p2  between 
the  solid  and  dashed  curves.  However,  just  increasing  p2 
to  10  dB  for  the  linear  system  suffices  to  obtain  the  same  Pe 
with  the  same  p2  =  12.5  dB,  and  so  the  limiter  affords 
only  a  2-dB  advantage  in  this  sense.  In  fact,  whenever  the 
larger  SNR  exceeds  the  smaller  by  more  than  3  dB,  it  would 
suffice  to  increase  just  the  smaller  one  for  the  linear  system 
by  less  than  3  dB.  When  the  larger  SNR  exceeds  the  smaller 
by  less  than  3  dB,  both  SNRs  must  be  increased  to  3  dB 
more  than  the  smaller. 

Springett  and  Simon  [12]  have  shown  that  for  a  hard- 
limiting  system,  in  the  absence  of  downlink  noise,  a  large 
SNR  improvement  p2  «  8  pf  is  achieved  at  large  values 
of  Pi2.  Unfortunately  this  improvement  in  p2  has  no 
bearing  on  the  error  rate. 

IV.  Optimum  Nonlinearity 

So  far  we  have  supposed  that  the  nonlinearity  is  a  hard 
limiter,  delivering  a  sine  wave  of  amplitude  a  to  the  receiver 
regardless  of  its  input  amplitude  R .  More  generally,  a  will 
be  a  function  of  R,  say  ah (AR/a^),  the  constant  factors  a 
and  A\a2  being  included  here  to  simplify  the  form  of  our 
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analysis.  To  take  account  of  this  nonlinearity  it  is  only 
necessary  to  replace  a  in  (1 1)  by  ah(ARjc2).  Using  (17)  for 
p((j))  in  (11)  and  substituting  r  =  AR\o2,  we  thus  get 

pe  =  ~~  exp  (~pi2)  r*  r 

2  SnpS  Jo  Jo 

.  ejp  [~(ri  erf  m cos  dr  4. 

(38) 


This  function  increases  linearly  from  zero  for  small  values 
of  r,  and  it  asymptotically  rises  toward  the  value  L  for  large 
r;  i.e.,  (41)  represents  a  sort  of  soft  limiter,  and  the  larger 
the  uplink  signal-to-noise  ratio,  the  harder  this  limiter  be¬ 
comes  on  account  of  the  factor  A/at2  in  the  argument  of  h. 
It  is  remarkable  that  (41)  is  the  same  nonlinearity  [13]  that 
maximizes  the  output  signal-to-noise  ratio  of  the  repeater 
output. 

More  generally,  we  see  from  (40)  that  h{ 0)  is  always  zero, 
that 


Since  r  has  a  Rice-Nakagami  distribution,  the  average 
repeater  power  delivered  to  the  receiver  is  the  mean  value 
of  -jC2A2(r) 

P,  =  exp  (-pj2)  f  r  exp  (-  -L_)  I0(r)h2(r)  dr. 

4Pi  Jo  V  4p,2/ 

(39) 

We  now  seek  that  h{r)  that  minimizes  (38)  under  the  con¬ 
dition  that  (39)  has  a  prescribed  value.  Treating  the  integrals 
with  respect  to  r  as  summations  over  discrete  values  of 
h(r ),  we  solve  this  problem  by  simply  differentiating  (38) 
and  (39)  with  respect  to  the  value  taken  by  h  at  one  specific 
value  of  r.  Introducing  the  Lagrange  multiplier  -1/ 
(\/2n  c2L)  for  the  derivative  of  (39),  setting  the  sum  of  this 
plus  the  derivative  of  (38)  equal  to  zero  and  cancelling  out 
the  common  factor 


exp  (~Pi2) 

4ft  \jln 


exp 


dr 


we  thus  obtain  [13] 


h(r)h(r) 

L  f2* 

=  —  exp  (r  cos  4>)  exp  (~±h2(r)  cos2  <p)  cos  $  d<j> 
2tt  J  o 

2 l  Cnl2 

=  —  sinh  (r  cos  <f>)  exp  (~±h2(r)  cos2  <t>)  cos  <f>  dd>. 

71  J  Q 

(40) 

Functions  h{r)  satisfying  (40)  can  be  found  by  solving  for 
L  and  determining  £  as  a  function  of  h  and  r.  The  contours 
over  the  ( h,r )  plane  along  which  L  is  constant  describe  the 
required  functions. 

To  determine  what  constant  value  L  must  have,  the 
resulting  h(r),  which  depends  on  L,  is  substituted  into  (39), 
and  L  is  found  as  a  function  of  p2  and  P,/<72  =  p22.  From 
(40)  we  see  that  h(r)  =  0  when  L  =  0  and  that  h(r)  is  an 
unbounded,  increasing  function  of  L  for  every  r.  It  follows 
from  (39)  that  L  =  0  when  p22  =  0  and  that  L  is  an 
unbounded,  increasing  function  of  p2. 

Although  the  solution  of  the  integral  equation  (40)  for 
h(r)  is  not  a  straightforward  matter,  it  can  be  solved 
analytically  in  extreme  cases.  For  example,  when  L  is  small, 
h  too  is  small,  and  the  second  exponential  in  the  first  line 
of  (40)  can  be  replaced  by  unity,  giving  us 

h{r)  =  £/,(/■  )//0(r).  (41) 


, ,  ,  1  .  3LJ  +  4L  3 

h(r)  =  -  Lr - r3  + 

2  64 


and  that,  as  r  -+  oo,  h(r)  approaches  the  value  satisfying 
the  equation 


K °o)  =  L  exp  [-i//2(oo)]  (42) 

as  the  integrand  of  (40)  then  peaks  at  (f>  =  0  and  the  right- 
hand  side  is  asymptotically 

Lexp  [— iA2(oo)]  •/,(»■). 


Thus,  whenever  the  uplink  signal-to-noise  ratio  is  very 
large,  a  hard  limiter  is  the  optimum  nonlinearity.  In  this 
case  we  can  determine  L  by  writing  (39)  in  the  form  p22  = 
%h2(co)  and  substituting  (42).  Hence,  for  large  pt2  we  have 

L  =  V2  p2  exp  (p22).  (43) 

When  L  is  large,  we  have 


h(r)  ^ 


(44) 


provided  that  (44)  is  large  in  comparison  with  r  +  1.  To 
see  this  notice  that  a  large  L  implies  a  large  h(r ),  which 
causes  the  integrand  of  (40)  to  be  negligible  except  near 
(j)  =  ^7r,  where  cos  (j>  &  —  (f>.  For  large  downlink 

signal-to-noise  ratios,  therefore,  the  optimum  nonlinearity 
(44)  rises  steeply  from  zero  for  small  r,  reaches  a  maximum 
where  r  =  1.61,  and  then  falls  toward  the  value  satisfying 
(42).  This  is,  in  fact,  qualitatively  the  same  as  the  output- 
versus-input  characteristic  of  a  traveling-wave  tube. 

To  see  that  the  h(r )  determined  by  (40)  must  yield  a 
minimum  rather  than  a  maximum  error  probability,  we 
notice  that  Pe  can  be  made  arbitrarily  close  to  unity  simply 
by  putting  h{r)  =  0,  for  all  r,  except  in  some  interval, 
which  is  allowed  to  shrink  to  zero  width.  Because  such  an 
h(r)  becomes  a  kind  of  delta  function  rather  than  a  proper 
function,  it  fails  to  show  up  in  our  solution,  and  so  we  ob¬ 
tain  only  the  optimum  h(r)  from  (40)  and  not  the  pessimum. 


V.  Conclusions 

General  expressions  have  been  derived  for  the  prob¬ 
ability  of  error  in  detecting  a  binary  phase-modulated 
sinusoidal  signal  after  transmission  through  a  hard-limiting 
channel.  The  receiver  employs  cross  correlation  followed  by 
zonal  low-pass  filtering  for  the  detection  of  the  signal.  The 
results  are  compared  with  those  for  a  linear  channel  and  a 
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coiYelation  receiver  not  using  postdetection  integration.  The 
error  rate  is  found  to  be  smaller  than  that  of  a  linear  channel 
for  the  range  of  SNRs  that  will  be  of  interest  in  practice.  In 
the  absence  of  either  repeater  or  receiver  input  noise  the 
error-rate  expression  is  identical  with  that  for  a  linear 
channel  with  no  postdetection  integration. 

The  results  of  this  paper  can  also  be  extended  as  in 
Lyons  [3]  to  include  the  effect  of  postdetection  integration 
via  multiple  sampling  and  majority  decision.  This  prob¬ 
ability  of  error  is  then  given  by  the  binomial  distribution 
into  which  we  need  only  to  substitute  the  error  probability 
for  a  single  sample  that  is  obtained  in  this  paper.  The  result 
is  a  monotonic  function  of  the  error  probability  for  a 
single  sample,  and  therefore  the  conclusions  reached  in  this 
paper  concerning  the  helpfulness  of  limiting  with  a  single 
sample  detection  also  apply  with  postdetection  integration. 

The  optimum  nonlinearity  has  been  determined  and  turns 
out  to  be  a  limiter  whenever  the  uplink  SNR  is  large. 
Remarkably,  this  optimum  nonlinearity  behaves  like  a 
traveling-wave  tube  when  the  downlink  SNR  is  large,  with 
its  output  falling  slowly  as  the  input  level  increases  beyond 
a  certain  value.  It  may  be  mentioned  that  limiting  can 
similarly  be  shown  to  reduce  the  error  probability  (although 
to  a  smaller  extent)  for  quadriphase  and  higher-order 
phase-shift-keying. 
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